Several formats have been presented for storing pixel data in a video subsystem. One approach is to provide twenty four bits of RGB information per pixel. This approach yields the maximum color space required for video at the cost of three bytes per pixel. Depending on the number of pixels in the video subsystem, the copy/scale operation could be overburdened by this.
A second approach is a compromise with the twenty four bit system. This approach is based on sixteen bits of RGB information per pixel. Systems of this nature require fewer bytes for the copy/scale operation but have the disadvantage of less color depth. Additionally, since the intensity and color information are encoded in the R, G and B components of the pixel, this approach does not take advantage of the human eye's sensitivity to intensity and insensitivity to color saturation. Other sixteen bit systems have also been proposed in which the pixels are encoded in a YUV format such as 6, 5, 5 and 8, 4, 4. Although these systems are somewhat better than the sixteen bit RGB approach, the sixteen bit YUV format does not come close to the performance of twenty bit systems.
Eight bit color lookup tables provide a third approach to this problem. This method uses eight bits per pixel as an index into a color map that typically has twenty bits of color space. This approach has the advantages of low byte count and while still providing twenty bit color space. However, there are only two hundred fifty six colors available on the screen in this approach and image quality may be somewhat poor.
Dithering techniques that use adjacent pixels to provide additional colors have been demonstrated to have excellent image quality, event for still images. However, these dithering techniques often require complicated algorithms and specialized palette entries in the digital-to-analog converter as well as almost exclusive use of the color lookup table. The overhead of running the dithering algorithm must be added to the copy/scale operation.
Motion video in some prior art systems is displayed in a 4:1:1 format called the "nine bit format". The 4:1:1 notation indicates that there are four Y samples horizontally for each UV sample and four Y samples vertically for each UV sample. If each sample is eight bits then a 4.times.4 block of pixels uses eighteen bytes of information or nine bits per pixel. Although image quality is quite good for motion video the nine bit format may be unacceptable for display of high-quality stills. In addition, it was found that the nine bit format does not integrate well with graphics subsystems. Other variations of the YUV subsampled approach include an eight bit format.
Systems integrating a graphics subsystem display buffer with a video subsystem display buffer generally fall into two categories. The two types of approaches are known as single frame buffer architectures and dual frame buffer architectures. The single frame buffer architecture is the most straightforward approach and consists of a single graphics controller, a single digital-to-analog converter and a single frame buffer. In its simplest form, the single frame buffer architecture represents each pixel on the display by bits in the display buffer that are consistent in their format regardless of the meaning of the pixel on the display. Thus, graphics pixels and video pixels are indistinguishable in the frame buffer RAM. However, the single frame buffer architecture graphics/video systems, i.e. the single frame buffer architecture visual system, does not address the requirements of the video subsystem very well. Full screen motion video on the single frame buffer architecture visual system requires updating every pixel in the display buffer thirty times every second. In a typical system the display may be on the order of 1280.times.1024 by 8 bits. Even without the burden of writing over 30 M Bytes per second to the display buffer, it has been established that eight-bit video by itself does not provide the required video quality. This means the single frame buffer architecture system can either move up to sixteen bits per pixel or implement the eight bit YUV subsampled technique. Since sixteen bits per pixel will yield over 60 M Bytes per second into the frame buffer, it is clearly an unacceptable alternative.
A visual system must be able to mix video and graphics together on a display which requires the display to show on occasion a single video pixel located in between graphics pixels. Because of the need to mix video and graphics within a display every pixel in the display buffer must be a stand-alone, self-sustaining pixel on the screen. The nature of the eight bit YUV subsampled technique makes it necessary to have several eight bit samples before one video pixel can be generated, making the technique unsuitable for the single frame buffer architecture visual system.
The second category of architecture which integrates video and graphics is the dual frame buffer architecture. The dual frame buffer architecture visual system involves mixing two otherwise free-standing single frame buffer systems at the analog back end with a high-speed analog switch. Since the video and graphics subsystems are both single frame buffer designs each one can make the necessary tradeoffs in spatial resolution and pixel depth with almost complete disregard for the other subsystem. Dual frame buffer architecture visual systems also include the feature of being loosely-coupled. Since the only connection of the two systems is in the final output stage, the two subsystems can be on different buses in the system. The fact that the dual frame buffer architecture video subsystem is loosely-coupled to the graphics subsystem is usually the overriding reason such systems, which have significant disadvantages, are typically employed.
Dual frame buffer architecture designs typically operate in a mode that has the video subsystem genlocked to the graphics subsystem. Genlocked in this case means having both subsystems start to display their first pixel at the same time. If both subsystems are running at exactly the same horizontal line frequency with the same number of lines, then mixing of the two separate video streams can be done with very predictable results.
Since both pixel streams are running at the same time, the process can be thought of as having video pixels underlaying the graphics pixels. If a determination is made not to show a graphics pixel, then the video information will show through. In dual frame buffer architecture designs, it is not necessary for the two subsystems to have the same number of horizontal pixels. As an example, it is possible to have 352 video pixels underneath 1024 graphics pixels.
The decision whether to show the video information or the graphics information in dual frame buffer architecture visual systems is typically made on a pixel by pixel basis in the graphics subsystem. A technique often used is called chroma keying. Chroma keying involves detecting a specific color in the graphics digital pixel stream or a specific color entry in the color lookup table. Another approach uses the graphics analog pixel stream to detect black, since black is the easiest graphics level to detect. This approach is referred to as black detect. In either case, keying information is used to control the high-speed analog switch and the task of integrating video and graphics on the display is reduced to painting the keying color in the graphics display where video pixels are desired.
There are several disadvantages to dual frame buffer architecture visual systems. The goal of high-integration is often thwarted by the need to have two separate, free-standing subsystems. The cost of having duplicate digital-to-analog converters, display buffers, and cathode ray tube controllers is undesirable. The difficulty of genlocking and the cost of the high-speed analog switch are two more disadvantages. In addition, placing the analog switch in the graphics path will have detrimental effects on the quality of the graphics display. This becomes a greater problem as the spatial resolution and/or line rate of the graphics subsystem grows.
A digital-to-analog converter is a key component in these visual frame buffer architectures. The digital-to-analog converter of these architectures accept both YUV color information and RGB color information simultaneously and provides chroma keying according to the received color information. In the prior art chroma keying systems a decision is made for each pixel of a visual display, whether to display a pixel representative of the YUV color value or a pixel representative of the RGB color value. The RGB value within a chroma keying system is typically provided by a graphic subsystem. The YUV value within a chroma keying system is typically provided by a video subsystem.
In these conventional chroma keying systems the determination regarding which pixel is displayed is based upon the RGB color value. Thus in a single display image there may be a mixture of pixels including both YUV pixels and RGB pixels. Thus it will be understood that each pixel displayed using conventional chroma keying systems is either entirely a video pixel or entirely a graphics pixel. Chroma keying merely determines which to select and provides for the display of one or the other. "Visual Frame Buffer Architecture", U.S. patent application Ser. No. 870,564, filed by Lippincott, and incorporated by reference herein, teaches a color lookup table method. In this method an apparatus for processing visual data is provided with storage for storing a bit plane of visual data in a one format. A graphics controller is coupled to the storage by a data bus and a graphics controller and the storage are coupled through a storage bus. Further storage is provided for storing a second bit plane of visual data in another format different from the first format. The further storage is coupled to the graphics controller by a data bus. The second storage is also coupled to the graphics controller through the storage bus. The method taught by Lippincott also merges a pixel stream from visual data stored on the first storage means and visual data stored on the further storage means. The merged pixel stream is then displayed.
Also taught in Lippincott is an apparatus for processing visual data including a first storage for storing a first bit plane of visual data in a first format. A graphics controller is coupled to the first storage means by a data bus, and the graphics controller and the first storage are coupled through a storage bus. A second storage for storing a second bit plane of visual data in a second format different from said first format is also provided. The second storage is coupled to the graphics controller by the data bus. The second storage is also coupled to the graphics controller through the storage bus. A merged pixel stream is formed from visual data stored on the first storage and visual data stored on the second storage. However this system is also adapted to provide only individual pixels which are entirely graphics or entirely video.
Referring now to FIG. 1, there is shown prior art visual frame buffer system 10. In visual frame buffer system 10 eight bit graphics pixels are received by way of YUV system input line 28 and applied to color lookup tables 32a-c within buffer system memory 30. Color lookup tables 32a-c typically contain two-hundred and fifty-six by eight bit maps. Within buffer system memory 30 of system 10 the pixel values accessed from table 32a are dedicated to red, the pixel values accessed from table 32b are dedicated to green, and the values accessed from table 32c are dedicated to blue.
It will be understood by those skilled in the art that a table lookup in buffer system memory 30, using an eight bit input pixel value, yields an eight bit table output value from each lookup table 32a-c. Thus a total of twenty four bits of graphics RGB information is provided from buffer system memory 30 onto RGB multiplexer input line 34 of pixel multiplexer 18. This permits simultaneously obtaining two-hundred fifty-six colors from graphics that are essentially twenty-four bits deep.
Pixel multiplexer 18 receives another twenty-four bits of RGB information within visual frame buffer system 10. This further twenty-four bits of RGB information is video information converted from a twenty-four bit YUV value. The YUV information is received by frame buffer system 10 by way of YUV system input line 24 and applied to YUV to RGB conversion matrix 14. A YUV to RGB conversion taught by Lippincott in "Minimal YUV/RGB Conversion Logic", copending with the present application, may be used for the purpose of efficiently converting from the YUV standard to the RGB standard as required within conversion matrix 14. However, it will be understood that other kinds of matrices effective to convert from YUV standard to RGB standard may be used within buffer system 10.
Thus on RGB multiplexer input line 16 pixel multiplexer 18 receives three eight bit RGB digital values corresponding to signals from video system input line 12, and on RGB multiplexer input line 34 pixel interpolator 18 receives three eight bit RGB digital values corresponding to graphics system input line 28. These signals may be applied to pixel multiplexer 18 as two twenty-four bit words. A selected one of these two twenty-four bit words on RGB multiplexer input lines 16, 34 is applied to digital-to-analog converter 22 by pixel multiplexer 18. The three eight bit values applied to digital-to-analog converter 22 by pixel multiplexer 18 are converted into three analog signals resenting the red, green and blue components of an image. The analog signals of converter 22 are applied to system output line 24 for display on a conventional color monitor.
In prior art visual frame buffer system 10 the section of a twenty-four bit input from the two twenty-four bit inputs of RGB multiplexer input lines 16, 34 by pixel multiplexer 18 is controlled by key compare device 36. Chroma key compare device 36 receives the twenty-four bit RGB value of line 34 which includes the outputs of color lookup tables 32a-c. Compare device 36 makes a determination whether to display a video pixel received by way of system input line 12 or a graphics pixel received by way of system input line 28 according to this value received on line 34. Chroma key compare device 36 controls pixel multiplexer 18 to select either RGB multiplexer input line 16 or RGB multiplexer input line 34 according to the pixel determination.
Control of multiplexer 18 may be accomplished by preprogramming compare device 36. For example, control of pixel multiplexer 18 may be triggered by red, blue or green values from lookup tables 32a-c which are equal to zero or two hundred fifty six. Thus, for example, when a programmed value of such as zero is determined to be present on line 34 by compare device 36, compare device 36 may cause pixel multiplexer 18 to apply converted video information to system output line 24 rather than graphics information from lookup tables 32a-c. However when performing these operations prior art visual frame system provides only output pixels which are either entirely graphics or entirely video.